Stein's unbiased risk estimate

In statistics, Stein's unbiased risk estimate (SURE) is an unbiased estimator of the mean-squared error of "a nearly arbitrary, nonlinear biased estimator."^[1] In other words, it provides an indication of the accuracy of a given estimator. This is important since the true mean-squared error of an estimator is a function of the unknown parameter to be estimated, and thus cannot be determined exactly.

The technique is named after its discoverer, Charles Stein.^[2]

Formal statement

Let $\mu \in {\mathbb R}^d$ be an unknown parameter and let $x \in {\mathbb R}^d$ be a measurement vector which is distributed normally with mean $\mu$ and covariance $\sigma^2 I$ . Suppose $h(x)$ is an estimator of $\mu$ from $x$ , and can be written $h(x) = x %2B g(x)$ , where $g$ is weakly differentiable. Then, Stein's unbiased risk estimate is given by

$\mathrm{SURE}(h) = d %2B \|g(x)\|^2 %2B 2 \sigma^2 \sum_{i=1}^d \frac{\partial}{\partial x_i} g_i(x),$

where $g_i(x)$ is the $i$ th component of the function $g(x)$ , and $\|\cdot\|$ is the Euclidean norm.

The importance of SURE is that it is an unbiased estimate of the mean-squared error (or squared error risk) of $h(x)$ , i.e.

$E_\mu \{ \mathrm{SURE}(h) \} = \mathrm{MSE}(h),\,\!$

with

$\mathrm{MSE}(h) = E_\mu \|h(x)-\mu\|^2.$

Thus, minimizing SURE can act as a surrogate for minimizing the MSE. Note that there is no dependence on the unknown parameter $\mu$ in the expression for SURE above. Thus, it can be manipulated (e.g., to determine optimal estimation settings) without knowledge of $\mu$ .

Applications

A standard application of SURE is to choose a parametric form for an estimator, and then optimize the values of the parameters to minimize the risk estimate. This technique has been applied in several settings. For example, a variant of the James–Stein estimator can be derived by finding the optimal shrinkage estimator.^[2] The technique has also been used by Donoho and Johnstone to determine the optimal shrinkage factor in a wavelet denoising setting.^[1]

References

^ ^a ^b Donoho, David L.; Iain M. Johnstone (December 1995). "Adapting to Unknown Smoothness via Wavelet Shrinkage". Journal of the American Statistical Association (Journal of the American Statistical Association, Vol. 90, No. 432) 90 (432): 1200–1244. doi:10.2307/2291512. JSTOR 2291512.
^ ^a ^b Stein, Charles M. (November 1981). "Estimation of the Mean of a Multivariate Normal Distribution". The Annals of Statistics 9 (6): 1135–1151. doi:10.1214/aos/1176345632. JSTOR 2240405.